Text input constitutes one of the most frequent computer user tasks. The QWERTY keyboard has been accepted as the standard tool for text entry for desktop computing. However, the emergence of handheld and other forms of pervasive or mobile computing calls for alternative solutions. These devices have small screens and limited keypads, limiting the ability of the user to input text. Consequently, text input has been revived as a critical research topic in recent years. The two classes of solutions that have attracted the most attention are handwriting and stylus-based virtual keyboarding.
Handwriting is a rather “natural” and fluid mode of text entry due to the prior experience of the user. Various handwriting recognition systems have been used in commercial products. However, the fundamental weakness of handwriting as a text entry method is its limited speed. While adequate for entering names and phone numbers, handwriting is too limited for writing longer text.
Virtual keyboards tapped serially with a stylus are also available in commercial products. The keyboard provided on the screen is typically the familiar QWERTY layout. Stylus keyboarding requires intense visual attention at every key tap, preventing the user from focusing attention on text output. To improve movement efficiency, optimization of the stylus keyboard layout has been considered both by trial and error and algorithmically. Using a keyboard layout such as ATOMIK (Alphabetically Tuned and Optimized Mobile Interface Keyboard), text entry is relatively faster. Reference is made to S. Zhai, M. Hunter & B. A. Smith, “Performance Optimization of Virtual Keyboards, Human-Computer Interaction,” Vol. 17(2, 3), 229-270, 2002.
The need for entering text on mobile devices has driven numerous inventions in text entry in recent years. The idea of optimizing gesture for speed is embodied in the Unistrokes alphabet. In the Unistrokes alphabet, every letter is written with a single stroke but the more frequent ones are assigned simpler strokes. If mastered, a user can potentially write faster in the Unistrokes alphabet than in the Roman alphabet. The fundamental limitation of the Unistrokes alphabet, however, is the nature of writing one character at a time. Reference is made to D. Goldberg, C. Richardsson, “Touch-typing with a stylus,” Proc. CHI. 1993, pages 80-87. Reference is also made to U.S. Pat. No. 6,654,496.
Quikwriting method uses continuous stylus movement on a radial layout to enter letters. Each character is entered by moving the stylus from the center of the radial layout to one of the eight outer zones, sometimes crossing to another zone, and returning to the center zone. The stylus trajectory determines which letter is selected. While it is possible to develop “iconic gestures” for common words like “the”, such gestures are relatively complex because the stylus has to return to the center after every letter. In this sense, the Quikwriting method is fundamentally a character entry method. Reference is made to K. Perlin, “Quikwriting: continuous stylus-based text entry,” Proc. UIST. 1998, pages 215-216.
A design has been proposed to rearrange the keyboard layout so that some common short words or word fragments can be wiped through, rather than pressed character by character. This method did not involve pattern recognition and shorthand production, so only characters in the intended word can be wiped through in the correct sequence. This design estimates that it would take about 0.5 motion per character. Reference is made to “Montgomery, E. B., “Bringing manual input into the 20th century: New Keyboard concepts,” Computer, Mar. 11-18, 1982, and U.S. Pat. No. 4,211,497.
Cirrin (Circular Input) operates on letters laid out on a circle. The user draws a word by moving the stylus through the letters. Cirrin explicitly attempts to operate on a word level, with the pen being lifted up at the end of each word. Cirrin also attempts to optimize pen movement by arranging the most common letters closer to each other. However, Cirrin is neither location nor scale independent. It does not uses word pattern recognition as the basic input method. Reference is made to J. Mankoff and G. D. Abowd, “Cirrin: a word-level unistroke keyboard for pen input”. Proc. UIST. 1998, pages 213-214.
It is important to achieve at least partial scale and location independency for the ease and speed of text entry. If all the letters defining a word on the keyboard have to be precisely crossed, the time and visual attention demand to trace these patterns is undesirable.
It is also important to facilitate skill transfer from novice behavior to expert performance in text entry by designing similar movement patterns for both types of behavior. The idea of bridging novice and expert modes of use by common movement pattern is used in the “marking menu”. Instead of having pull-down menus and shortcut keys, two distinct modes of operation for novice and expert users respectively, a marking menu uses the same directional gesture on a pie menu for both types of users. For a novice user whose action is slow and needs visual guidance, marking menu “reveals” itself by displaying the menu layout after a pre-set time delay. For an expert user whose action is fast, the marking menu system does not display visual guidance. Consequently, the user's actions become open loop marks. However, the marking menu is not used for text entry due to the limited number of items can be reliably used in each level of a pie menu (8 or at the most 12). Reference is made to G. Kurtenbach, and W. Buxton, “User Learning and Performance with Marking Menus,” Proc. CHI. 1994, pages 258-264; and G. Kurtenbach, A. Sellen, and W. Buxton, “An Empirical Evaluation of Some Articulatory and Cognitive Aspects of “Marking Menus”, Human Computer Interaction, 1993, 8(1), pages 1-23.
A self-revealing menu approach, T-Cube, defines an alphabet set by cascaded pie menus that are similar to a marking menu. A novice user enters characters by following the visual guidance of menus, while an expert user can enter the individual characters by making menu gestures without visual display. A weakness of the T-Cube is that it works at alphabet level; consequently, text entry using T-cube is inherently slow. Reference is made to D. Venolia and F. Neiberg, “T-Cube: A fast, self-disclosing pen-based alphabet”, Proc. CHI. 1994, pages 265-270.
Dasher, another approach using continuous gesture input, dynamically arranges letters in multiple columns. Based on preceding context, likely target letters appear closer to the user's cursor location. A letter is selected when it passes through the cursor; consequently, cursor movement is minimized. This minimization, however, is at the expense of visual attention. Because the letter arrangement constantly changes, Dasher demands user's visual attention to dynamically react to the changing layout. Reference is made to D. J. Ward, A. F. Blackwell, and D. J. C. MacKay. “Dasher—a data entry interface using continuous gestures and language models”, Proc. UIST. 2000, pages 129-137.
One possibility for introducing gesture-based text entry is the use of shorthand. Traditional shorthand systems are efficient, but hard to learn for the user and difficult to recognize by the computer. Shorthand has no duality; it cannot be used by experts and beginners alike. In addition, shorthand has no basis in a virtual keyboard, so the user cannot identify the required symbol from the keyboard. If the user forgets shorthand symbols, a separate table must be consulted to find the symbol.
One recent approach comprises a form of continuous gesture-based text input that requires minimal visual attention. This approach is based on keyboard entry, recognizing word patterns based on a virtual keyboard layout. Handwriting recognition is combined with a virtual, graphical, or on-screen keyboard to provide a text input method with relative ease of use. The system allows the user to input text quickly with little or no visual attention from the user. Reference is made to Zhai, S. and Kristensson, P.-O., “Shorthand Writing on Stylus Keyboard,” CHI 2003, ACM Conference on Human Factors in Computing Systems, CHI Letters 5(1), (Fort Lauderdale, Fla., 2003), ACM.”
Although this gesture-based text input system has proven to be useful, it is desirable to present additional improvements. A gesture-based text input system with a larger vocabulary of gestures recognizable as words is desired. In addition, a method for identifying the appropriate word among the larger gesture vocabulary is desired. Further, a method for enhancing user learning that accelerates the acquisition and use of gestures to speed text input is desired. The need for such system and method has heretofore remained unsatisfied.